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During  the  reporting  period  rye  studied  the  interchangeability/ 
Isubstitutability  problem  for  two  Recoverable  items  that  fail  at  a single  locatioK 
We  assumed  the  failure  processes  for  each  type  of  item  are  independent,  stations): 
Poisson  processes  and  that  the  repair  times  are  exponentially  distributed. 
Furthermore,  we  assumed  that  the  system  is  a closed  system,  that  is,  no  items 
are  added  to  or  deleted  from  the  system.  We  first  considered  a discrete-time 
problemand  showed  that  this  problem  Is  a Markovian  decision  problem.  We 
then  showed  that  for  this  problem  there  exist  optimal  stationary  Markov 
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20  Abstract  (continued) 

^control  policies.  Next  we  formulated  a continuous  time  model  and  showed  how  to 
find  the  optimal  stationary  Markov  control  policy  using  linear  programming. 
Unfortunately,  this  approach  was  shown  to  be  impractical  for  solving  most  real 
problems.  Consequently,  we  established  and  explored  some  of  the  properties 
that  we  feel  an  optimal  policy  should  possess.  We  developed  heuristic  procedures 
that  can  be  used  to  find  a good  policy  for  managing  this  class  of  items. 
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During  the  reporting  period  Professors  David  Heath,  John  Muckstadt, 
and  Carol  Shilepsky  have  conducted  research  on  the  interchangeability/ 
substitutability  problem  for  recoverable  items  (items  that  are  subject 
to  repair  when  they  fail).  This  problem  arises  when  recoverable  items 
can  be  substituted  for  one  another  during  the  repair  of  an  assembly. 

The  long  range  objective  of  the  research  project  is  to  develop  a 
method  that  the  Air  Force  can  use  to  assist  in  the  management  of  inter- 
changeable recoverable  items.  To  accomplish  this  goal  we  have  proposed 
to  analyze  several  simplified  problems  that  will  give  us  insight  into 
the  form  of  the  optimal  or  near  optimal  policy  for  the  real  problem. 
Specifically,  we  proposed 

a)  to  understand  fully  the  behavior  of  a single-echelon,  two  item 
system  over  an  infinite  horizon  when  the  failure  processes  for  the  items 
are  independent,  stationary  Poisson  processes  and  repair  times  are  expo- 
nential; 

b)  to  develop  methods  for  finding  optimal  and  near  optimal  policies 
for  the  situation  described  in  a); 

c)  to  extend  the  results  for  topics  a)  and  b)  to  systems  having  many 
items; 

d)  to  extend  the  analysis  to  situations  in  which  there  are  two  echelons 
(depot-base  structure)  and  many  items  where,  as  before,  the  failure  processes 
for  the  item  are  independent,  stationary  Poisson  processes  and  repair  times 
are  exponential; 

e)  to  develop  methods  for  finding  the  optimal  or  near  optimal  operating 
policy  for  the  situation  described  in  d); 
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f)  to  study  the  problem  when  failure  and  repair  processes  need  not 


be  stationary  and  the  time  horizon  is  finite  for  both  single  and  two 
echelon  systems;  and 

g)  to  develop  heuristic  dispatching  rules  for  the  dynamic  environment 
described  in  f). 

To  date  we  have  almost  completed  objectives  a)  and  b).  Specifically, 
we  have  studied  the  interchangeability/substitutability  problem  for  two 
items  that  fail  at  a single  location.  Moreover,  the  failure  processes 
for  these  items  are  assumed  to  be  independent,  stationary  Poisson  processes, 
and  repair  times  are  exponentially  distributed.  The  system  studied  is 
assumed  to  be  a closed  system,  that  is,  no  items  are  added  to  or  deleted 
from  the  system.  Based  on  these  assumptions  we  first  showed  that  the  prob- 
lem could  be  viewed  as  a Markovian  decision  problem  for  which  there  exists 
a stationary  optimal  policy.  Since  our  main  goal  is  to  find  methods  that 
can  be  used  to  solve  the  real  problem,  we  next  developed  various  approaches 
for  finding  optimal  and  near  optimal  policies. 

The  first  approach  that  we  took  was  to  formulate  the  decision  problem 
as  a linear  programming  problem.  The  number  of  "states"  in  the  Markovian 
decision  problem  for  realistic  problems  is  so  large  that  the  linear  program- 
ming method  is  not  a practical  method  for  finding  optimal  policies.  It  is, 
however,  useful  as  an  experimental  tool  to  check  the  optimality  or  near 
optimality  of  solutions  generated  by  other  means.  Furthermore,  we  were  able 
to  identify  some  properties  that  an  optimal  policy  should  possess. 

Next,  we  developed  a simulation  approach  that  exploits  these  properties 
that  an  optimal  policy  is  conjectured  to  have.  This  heuristic  method  is 
computationally  efficient  and  finds,  at  least  for  the  cases  tested,  a nearly 


optimal  policy.  (The  details  of  the  results  of  this  work  can  be  found 
in  the  attached  technical  report,  entitled  "An  Analysis  of  a Single 
Location  Inventory  Problem  for  Two  Interchangeable  Recoverable  Items.") 
Recently,  a new  heuristic  has  been  developed  which  may  be  even  more  ef- 
ficient. Further  testing  of  this  method  is  currently  being  conducted. 

Throughout  our  analysis  we  have  been  in  contact  with  Lt.  Col.  Jon  Hobbs 
and  Mr.  Victor  Presutti  cf  Headquarters,  Air  Force  Logistics  Command  (AFLC/ 
XRS).  Their  guidance  has  been  extremely  valuable.  They  are  providing  data 
that  will  be  used  to  test  the  proposed  heuristics.  Once  these  test  are  com- 
pleted and  adjustments  made  to  the  heuristic  we  will  have  completed  goals 
a)  and  b)  stated  above. 

We  have  been  invited  and  will  present  the  results  of  our  research  at 
the  International  Meeting  of  The  Institute  of  Management  Sciences  on 
June  20,  1979.  Furthermore,  we  plan  to  prepare  a paper  having  the  same 
title  as  the  attached  technical  report  and  submit  it  for  publication  in 
the  Naval  Research  Logistics  Quarterly. 
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ABSTRACT 


In  this  paper  we  examine  the  interchangeability/substitutability 
problem  for  two  recoverable  items  that  fail  at  a single  location.  We 
assume  the  failure  processes  for  each  type  of  item  are  independent, 
stationary  Poisson  processes.  We  also  assume  the  repair  times  are 
exponentially  distributed.  Furthermore,  we  assume  that  the  system  is 
a closed  system,  that  is,  no  items  are  added  to  or  deleted  from  the 
system.  We  first  consider  a discrete-time  problem  and  show  that  this 
problem  is  a Markovian  decision  problem.  We  then  show  that  for  this 
problem  there  exist  optimal  stationary  Markov  control  policies . Next 
we  formulate  a continuous  time  model  and  show  how  to  find  the  optimal 
stationary  Markov  control  policy  using  linear  programming.  Unfortunately, 
this  approach  is  impractical  for  solving  most  real  problems.  Consequently 
we  have  established  and  explored  some  of  the  properties  that  we  feel 
an  optimal  policy  should  possess.  A discussion  of  these  properties  is 
given  in  Section  IV.  Lastly,  we  will  describe  a heuristic  that  can  be 
used  to  find  a good  policy.  This  method  is  an  efficient  simulation  search 
method  that  finds  policies  having  the  properties  we  conjecture  an 
optimal  policy  should  possess. 


I . INTRODUCTION 


During  the  past  15  years  a substantial  amount  of  research  has  been 


conducted  related  to  the  management  of  recoverable  items,  that  is,  items 
subject  to  repair  when  they  fail  >»>»»»>>»  .A 

number  of  mathematical  models  have  been  developed  that  can  be  used  to 
determine  optimal  stockage  levels  for  each  recoverable  item  in  both  single 
and  multi-echelon  systems.  Most  of  the  models  are  based  on  the  assumption 
that  the  items  are  independent.  That  is,  the  failure  processes  among  the 
items  are  assumed  to  be  independent.  Some  recent  research  has  been  devoted 
to  dependencies  in  the  demand  process  by  recognizing  that  certain  recoverable 
items  have  a hierarchical  design  ^1*5,6,13,14].  por  these  items,  the 
failure  of  a recoverable  component  results  in  a demand  for  both  a spare 
component  and  the  assembly  containing  the  component.  However,  in  all  of 
the  models  presented  to  date  the  replacement  rule  for  a failed  component  is 
the  same:  replace  all  failed  units  with  a serviceable  spare  item  of  the 
same  type. 

In  this  paper  we  examine  a problem  that  arises  when  items  are  some- 
times interchangeable  or  can  be  substituted  for  one  another  during  repair. 
Frequently  it  is  possible  to  repair  a broken  assembly  using  several  differ- 
ent types  of  parts;  however,  choosing  the  "correct"  part  to  use  to  repair 
the  assembly  is  not  based  on  engineering  considerations  alone.  Using  one 
type  of  item  to  complete  the  repair  rather  than  using  a second  type  of  item 
can  cause  subsequent  parts  shortages  that  can  be  avoided.  This  can  occur 
because  some  items  are  "more  useful"  than  others.  For  example,  suppose 
there  are  only  two  types  of  items  in  the  system.  The  "more  useful"  item 
can  be  used  to  satisfy  a demand  for  either  type  of  item  whereas  the 
"less  useful"  item  can  only  be  used  to  satisfy  demands  for  its  own  type 
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of  item.  Many  such  interchangeable/substitutable  items  are  found  in  the 
Air  Force.  This  is  particularly  the  case  for  electronic  items.  In  some 
instances,  newly  designed  items  can  be  used  to  replace  older  units  when  they 
fail;  however,  these  older  units  cannot  be  used  to  repair  a newer  generation 
of  an  assembly. 

In  this  paper  we  will  examine  the  interchangeability/substitutability 
problem  for  two  items  that  fail  at  a single  location.  We  assume  the  failure 
processes  for  each  type  of  item  are  independent,  stationary  Poisson  processes. 
We  also  assume  the  repair  times  are  exponentially  distributed.  Furthermore, 
we  assume  that  the  system  is  a closed  system,  that  is,  no  items  are  added 
to  or  deleted  from  the  system.  We  first  consider  a discrete-time  problem 
and  show  that  this  problem  is  a Markovian  decision  problem.  We  then  show 
that  for  this  problem  there  exist  optimal  stationary  Markov  control  policies. 
Next  we  formulate  a continuous-time  model  and  show  how  to  find  the  optimal 
stationary  Markov  control  policy  using  linear  programming.  Unfortunately, 
this  approach  is  impractical  for  solving  most  real  problems.  Consequently 
we  have  established  and  explored  some  of  the  properties  that  we  feel  an 
optimal  policy  should  possess.  A discussion  of  these  properties  is  given 
in  Section  IV.  Lastly,  we  will  describe  a heuristic  that  can  be  used  to 
find  a good  policy.  This  method  is  an  efficient  simulation  search  method 
that  finds  policies  having  the  properties  we  conjecture  an  optimal  policy 
should  possess. 


II.  A TWO-ITEM  SINGLE  LOCATION  SYSTEM: 

A DISCRETE  TIME  MODEL 

In  this  section  we  present  a discrete-time  model  for  the  inter- 
changeability/substitutability problem.  We  examine  this  discrete-time 
model  since  it  yields  a particular  form  for  the  optimal  control  policy. 

In  particular,  we  show  that  for  the  associated  Markov  decision  problem 
there  is  a stationary  Markov  control  policy  which  achieves  the  lowest 
average  back-order  level.  This  result  provides  the  motivation  for 
restricting  attention  to  Markov  control  policies  in  the  continuous  time 
model  developed  in  the  next  section. 

To  simplify  the  analysis  and  discussion  we  restrict  our  attention 
to  a single  location  system  with  only  two  types  of  items:  type  1 and 
type  2.  The  assemblies  in  which  these  items  are  installed  are  called  units; 
we  also  assume  that  there  are  two  types  of  units  and  that  each  unit  contains 
only  one  item  of  the  types  considered.  Furthermore,  when  a type  1 unit  or  a 
type  2 unit  fails,  we  assume  that  it  can  be  repaired  with  a serviceable 
type  2 item.  Type  1 items  can  also  be  used  to  repair  failed  type  1 units: 
howeverj type  1 items  cannot  be  used  to  repair  type  2 units.  For  example, 
two  units  might  be  different  "generations"  of  a computer  found  in  a fire 
control  system;  the  items  might  be  old  and  new  versions  of  an  integrated 
circuit  board  found  in  the  computer.  The  newer  version  of  the  circuit  board 
can  be  used  in  both  generations  of  the  fire  control  system  computer;  but, 
the  old  generation  circuit  board  is  incompatible  with  the  newer  fire  control 
system  computer. 

Let  Nj  be  the  number  of  units  of  type  i and  M^  be  the  number 
of  spare  items  of  type  i in  the  system.  Thus  there  are  a total  of 
IT  ♦ Mj  items  in  the  system.  Let  n^  be  the  number  of  type  i items 
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installed  in  type  j units,  and  nu  be  the  number  of  servicable  type  i 
items  in  spare  stock. 

Note  that  according  to  the  substitution  rules  we  have  established 

n^2  is  always  zero.  Thus  the  five  numbers  m^  , m^  , n^  , n^  , and 

n specify  the  disposition  of  all  items.  The  number  of  type  i items 

i 

in  repair  is  given  by  N.+M.-(m.+  E n..),  and  the  number  of  back- 

3 2 

orders  associated  with  type  j units  is  N.  - I n.  . . 

! • • 1 j 

i=: 


We  assume  the  system  operates  as  follows  in  the  discrete  time  model. 

At  each  time  t=n  • (At),n=0,  1,  ...  certain  actions  are  available. 
These  actions  correspond  to  installing  some  items  currently  in  spare  stock 
in  appropriate  units  lacking  an  item.  After  installation  of  items, 
failures  may  occur.  We  presume  that  items  fail  independently  of  one  another 
and  that  each  item  of  type  i installed  in  a unit  of  type  j has  proba- 
bility • (At)  of  failing  (where  (At)  is  small  enough  so  that  these 

numbers  do  not  exceed  1).  Next,  items  which  have  failed  are  removed  from 
the  units  and  sent  to  repair.  Items  are  repaired  independently;  we  presume 
that  each  item  is  repaired  during  this  time  interval  with  probability 
• (At)  . 

After  this  sequence  of  action-failure-repair  we  begin  again  at  the  new 
time  (n+1)  • At  by  selecting  another  action.  The  system  continues  to 
operate  in  this  manner  for  an  indefinite  length  of  time. 

The  number  of  backorders  during  the  action-failure-repair  cycle  is 
defined  to  be  the  number  of  backorders  which  exist  immediately  after  the 
action  (unit  repair)  is  taken  and  before  the  failures  occur. 

We  wish  to  choose  those  actions  which  minimize  the  average  number 
of  backorders. 
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The  selection  of  the  particular  sequencing  of  events  - action,  failure 
repair  - was  not  made  arbitrarily.  This  sequence  was  selected  so  that  the 
problem  could  be  formulated  as  a Markovian  decision  problem  (average  cost 
model).  (A  discussion  of  Markovian  decision  problems  can  be  found  in 
reference  10 . ) 

Notice  that  for  any  policy  (that  is,  a specification  of  the  actions  to 
be  taken  for  all  possible  situations)  there  is  positive  probability  (actually 
bounded  away  from  zero)  that  after  one  cycle  of  action-failure-repair  all 
items  which  were  in  use  will  have  failed  and  been  repaired.  Hence  there 
is  a state  (namely  that  with  no  items  installed  and  all  in  spare  stock) 
for  which  evary  action  taken  at  every  state  gets  to  that  state  in  one  step 
with  probability  greater  than  or  equal  to  g > 0 . By  Ross  [10]  (Theorem 
6.17,  Corollary  6.20,  and  the  remarks  following  Corollary  6.20)  there  is 
then  an  optimal  stationary  Markov  control  policy.  This  policy  can  be  com- 
puted by  a technique  involving  linear  programming;  this  technique  is  adapted 
to  the  continuous -time  model  developed  in  Section  III. 

I 
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III.  A TWO-ITEM  SINGLE  LOCATION  SYSTEM: 

A CONTINUOUS  TIME  MODEL 

We  now  consider  the  continuous  time  model  corresponding  to  that  of 
Section  II.  The  notation  describing  the  numbers  of  units  and  items  and 
the  state  of  the  system  remain  the  same. 

We  now  suppose  that  the  failure  times  of  type  i items  installed  in 
type  j units  are  independent  exponentially-distributed  random  variables 
with  mean  1/A^  » and  that  the  repair  times  are  independent,  exponential 
random  variables  with  mean  1/r^  • As  before,  the  measure  of  performance 
of  the  system  is  the  average  number  of  backorders.  Motivated  by  the 
results  of  the  previous  section  we  shall  consider  only  stationary  Markov 
control  policies. 

Since  it  would  seem  unreasonable  to  allow  backorders  for  type  i units 
if  there  are  type  i items  in  spare  stock  we  consider  the  installation  of 
type  i items  in  type  i units  to  be  automatic  and  not  subject  to  control. 

Thus  the  actions  available  involve  only  the  installation  of  type  2 items 
in  type  1 units.  We  shall  compute  the  optimal  stationary  Markov  control 
policy  which  takes  action  only  when  the  system  changes  state  due  to  an  item 
failing  in  service  or  being  returned  from  repair.  Finally,  we  allow  only  the 
substitution  of  one  type  2 item  into  type  1 units  at  a time. 

When  the  process  jumps  to  a new  state  there  are  (possibly)  two  actions 


I 

L 


available:  do  nothing,  or  install  a type  2 item  in  a type  1 unit.  Of  course, 

if  there  are  no  type  2 items  available  or  no  backorders  associated  with 

type  1 units,  then  there  is  only  one  action  available:  do  nothing. 

Following  Ross  [10 j we  allow  randomized  actions;  thus  corresponding  to 

ai  a2 

state  S there  are  two  numbers  Pg  and  (non-negative  and  summing  to  1) 

giving  the  probabilities  of  selecting  action  a^  (=  do  nothing)  or  a2 
(=  put  one  type  2 item  in  a type  1 unit).  These  P's  completely  specify  the 
control  policy. 


Once  these  P's  are  specified  the  process  which  results  is  a stationary 
Markov  process.  In  fact,  if  we  consider  the  process  which  specifies  the  state 
the  system  most  recently  jumped  to  and  the  action  taken  there,  this  process  is 


a Markov  chain  and  we  can  compute  its  stationary  transition  probabilities 
and  then  the  average  cost . 


Let  Zg  be  the  equilibrium  (or  stationary)  probability  that  this  process 


most  recently  jumped  to  state  S and  that  the  action  taken  there  was  a . Th 


we  must  have  E E „ Z^  = E Z^  where  a _ (S  ^S0)  is  the  rate  at 
a S2  Slb2  S2  a S1  blb2  1 Z 


which  transitions  to  state  will  occur  if  the  current  state  is  and 


the  action  taken  is  "a"  , and  = -E 

b2b2  S T2 


so  that  the  column-suns 


(Sj*  s2) 


of  the  a-matrix  are  zero.  (The  subscripts  may  seem  reversed  in  the  above. 
This  is  because  in  the  usual  Markov  chain  matrix  notation  the  order  of 
vector-matrix  multiplication  is  the  reverse  of  that  used  in  the  usual  L.P. 
notation,  which  we  adopt  here.) 

The  cost  associated  with  each  state,  C(S,a)  , is  the  number  of  back- 
orders (total  for  units  of  type  1 and  type  2)  associated  with  state  S if 
action  a is  chosen. 


We  wish  to  minimize  the  cost-per-unit-time  given  by  E E Z C(S,a) 

S a b 


subject  to  the  equilibrium  equations  EE  Z^  = E Z^  and  to  the 

a S bl  2 b2  a bl 

condition  E E Z^  = 1 
S a b 


zs  > ° . 


(Note  that  in  some  states  substitution  obviously  cannot  be  performed; 
for  these  states  we  can  simply  ignore  (that  is  set  to  zero)  the  corresponding 


ZT  for  a = SUBSTITUTE). 


mid'  i’^ii  tfl  »J*1  ..<■■■  ii 
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The  solution  to  the  above  stated  L.P.  provides  the  values  for  the  P's 
(and  hence  the  policy)  as  follows: 

P^  = P{take  action  a when  in  state  S) 

. A.. 

E Z* 


whenever  E / 0 . When  I = 0 , state  i is  never  reached  by  the 
b b b b 


controlled  system  and  thus  it  makes  sense  to  leave  the  action  chosen  there 
undefined. 

Suppose  T represents  the  number  of  realizable  states  in  the  system 

(a  state  is  realizable  whenever  E Zg  > 0 ).  Then  the  above  optimization 

a 

problem  has  T equality  constraints  and  T basic  variables  (note  there  is 
one  redundant  constraint).  If  state  S is  realizable,  then  Z^  > 0 for  at 
least  one  action  a . Consequently  at  least  one  is  positive  for 

S=l,  ...,  T . This  implies  that  for  each  state  S , Z^  can  be  positive  for 

only  one  action  a . Thus  P^  must  be  either  0 or  1 . 

A program  was  written  to  generate  the  input  for  a linear  programming 
package  (MPSX).  The  output  was  then  submitted  to  MPSX  and  (in  almost  all 
cases)  an  optimal  strategy  was  obtained. 

To  present  the  results  of  the  computations  we  notice  that  under  our 

control  policies  the  number  of  variables  necessary  to  describe  the  state  of 

the  system  can  be  reduced  from  the  five  above  to  three  as  follows:  Since 
we  never  allow  backorders  for  type  i units  if  there  is  serviceable  spare 


stock  on  hand; 


N. 

l 


2 

- E 

j=i 


n. . 
It 


and  m. 
i 


cannot  both  be  positive. 


We  thus  set 


s.  = m.  - [N.  - E n..],  which  is  the  net  inventory  of  items  of  type  i 

i 1 •11  — — *—  * 

1=1  J 

(it  is  negative  if  there  are  backorders  associated  with  type  i units). 
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The  variables  and  then  describe  the  state  of  the  system. 

Finally,  since  at  each  state  the  control  policy  merely  specifies 
"substitute"  or  "don't  substitute"  it'  suffices  to  graph  the  set  of  states  S 
at  which  one  would  perform  a substitution.  Graphs  of  the  optimal  "substitution 
set"  S for  several  situations  are  given  in  Figure  1,  Figure  2,  and  Figure  3. 
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Example  A.  Optimal  Policy 
Figure  1 . 
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IV.  THE  FORM  OF  AN  OPTIMAL  POLICY 

To  enable  us  to  find  approximations  to  an  optimal  policy,  we  examined 
the  structure  of  the  set  of  states,  S , from  which  a type  2 item  should 
be  substituted  into  a type  1 unit.  Clearly 

S - {(sx,  s2,  n21)  : sx  < 0,  s2  > 0}  . 

Every  such  subset  gives  rise  to  a control  policy;  however,  there  are 
three  properties  (monotonicity  relationships)  that  one  might  expect  the 
optimal  subset  (S  to  have.  Willingness  to  perform  a substitution 

depends  on  the  number  of  type  1 units  out  of  service,  the  number  of  type  2 
items  in  spare  stock,  and  the  number  of  type  2 items  already  installed  in  type 
1 units.  Intuitively,  for  fixed  s2  and  n2x  , as  s^  decreases  there  is 
at  least  as  great  a need  to  substitute  type  2 parts  in  the  type  1 units. 

We  express  this  as 

MR1)  (s1,  s 2 , n21)  e Sopt  , s < sx  , implies  (s,  s2>  n21>  e Sopt  . 
Similarly,  a greater  supply  of  type  2 spares  should  imply  an  equal  or  greater 
willingness  to  substitute.  This  property  is  expressed  as 


MR2)  (sx,  s2,  n2J  t S , s > s2  , implies  ( s x>  s,  n^)  e S£ 


Finally , 


MR3)  (sx,  s2,  n21)  e Sopt  , n < n21  , implies  (s^  s2,  n)  e SQpt  . 

Since  S is  a set  in  three-dimensional  space  we  can  draw  its  graph  in 
sections;  thus  if  we  let  each  section  correspond  to  a fixed  value  of  s^  we 
would  expect  the  graph  to  look  like  that  of  Figure  4. 

In  each  optimal  strategy  computed  using  the  linear  programming  method 
described  in  Section  III  the  properties  MR1)-MR3)  held.  On  the  basis  of 
this  conjectured  form  of  the  optimal  policy  we  developed  a procedure  to 
examine  states  for  possible  inclusion  in  S . 
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V.  APPROXIMATIONS  TO  AN  OPTIMAL  POLICY 

To  find  a good  approximation  to  the  optimal  substitution  policy  we 
employed  a search  technique  which  proceeds  as  follows:  begin  with  S 
empty  (i.e.,  with  a policy  which  allows  no  substitution);  repeatedly 
consider  adding  one  point  at  a time  to  S by  comparing  the  performance 
of  the  system  using  the  augmented  S with  the  current  policy;  then  add 
the  point  if  the  level  of  performance  is  higher. 

The  first  states  considered  are  those  for  which  s^  = -1  . In  this 
set,  the  most  likely  candidate  for  membership  in  S is  (-1,  M^,  0): 
i.e.,  allow  substitution  when  there  is  a type  1 unit  out  of  service  only 
if  there  are  M^  type  2 items  in  stock  and  none  already  installed  in 
type  1 units.  If  this  state  is  added  to  S , s2  is  successively  decreased 
by  one  unit  as  long  as  the  inclusion  of  (-1,  s2,  0)  improves  the  policy. 

Then  n21  is  incremented  by  1 and  (-1,  M2~l,  1)  is  considered  (note, 
if  n21  = 1 , then  at  most  M2~l  serviceable  spare  type  2 items  can  be  in 
stock).  Again,  s2  is  decreased  until  there  is  no  further  policy  improvement. 
We  then  increment  n21  again.  After  all  appropriate  states  of  the  form 
(-1,  s2,  n2^)  have  been  added  to  S , those  for  which  s^  = -2  are  con- 
sidered in  a similar  fashion.  We  take  advantage  of  property  MR1)  to  include 
automatically  in  S each  state  (-2,  s2>  n21)  such  that  (-1,  s2,  n21)  has 
already  been  added  to  S . This  significantly  reduces  the  number  of  comparisons 
which  have  to  be  made. 

An  alternate  search  pattern  was  considered  in  which  (-N^,  M2,  0)  is 
the  first  state  examined  for  inclusion  in  S . In  this  state  all  type  1 units 
are  out  of  service  and  all  type  2 items  are  in  spare  stock.  Intuitively, 


this  is  the  state  from  which  one  would  be  most  likely  to  allow  substitution 
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and  hence  an  appropriate  starting  point  for  the  search.  The  disadvantage 
of  this  approach  was  that  MR1)  could  not  be  used  to  increase  the  computational 
efficiency,  and  hence  the  first  method  was  used. 

The  search  technique  involved  being  able  to  compare  the  performance  of 
two  policies.  We  developed  two  methods  for  this:  one,  analytic,  which  gave 
an  exact  number  for  the  expected  backorders  under  a given  policy,  and  the 
second,  simulation. 

In  the  analytic  method  we  observe  that  the  substitution  rules  for  a given 
policy  and  the  transition  probabilities  depend  only  on  the  present  state, 
hence  the  system  is  a Markov  chain.  The  method  consists  of  generating  the 
steady  state  equations , finding  the  equilibrium  probability  distribution 
for  the  chain  and  calculating  the  expected  number  of  backorders  under  the 
equilibrium  distribution.  This  number  can  be  compared  to  the  expected  back- 
orders for  the  same  policy  augmented  by  one  state  as  required  by  the  search 
procedure . 

A serious  disadvantage  of  this  method  of  policy  comparison  is  related 
to  the  number  of  states  in  the  chain  and  hence  the  number  of  equations  to  be 
solved  for  the  equilibrium  distribution.  The  number  of  .states  for  a system 
with  units  of  type  i and  items  of  type  i is 

(N.  + M.  + 1)(M2  + 1)(N2  + M2/2  + 1)  , 

For  example,  a relatively  small  system  with  N..  = 10  and  = 5 has  1248 
states.  When  the  number  of  units  and  items  grows  by  a factor  of  n , the 

3 

number  of  states  increases  roughly  by  a multiple  of  n . The  time  required 

to  solve  the  equations  corresponding  to  the  enlarged  system  then  increases 

9 

by  approximately  a multiple  of  n . We  are,  therefore,  critically  limited 
in  the  size  of  the  system  we  can  investigate  using  the  Markov  analysis  to 
compare  policies. 
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The  major  advantage  of  this  approach  is  that  the  answers  are  exact  and 
hence  two  policies  can  be  compared.  The  similar  results  obtained  by  linear 
programming  and  by  the  search  technique  with  analytic  policy  comparison 
suggested  that  the  search  mechanism  is  valid  and  encouraged  the  development 
of  a more  efficient  method  of  policy  comparison. 

The  simulation  method,  which  provides  the  only  tool  suitable  for  the 
analysis  of  large  systems,  performs  two  simulations,  one  for  each  policy 
and  uses  the  state  at  which  the  policies  differ  as  a starting  point  for  the 
simulation.  In  one,  the  substitution  of  a type  2 part  is  made,  and  in  the 
other  it  is  not  made.  The  two  simulations  are  then  run  until  they  both 
reach  the  same  state.  This  state  is  not  necessarily  the  one  in  which  the 
simulations  started.  If  either  system  reaches  the  initial  state  in  which 
the  desirability  of  substitution  is  being  questioned,  the  substitution  is 
not  made. 

When  the  two  systems  reach  the  same  state  the  run  is  terminated  and  the 

difference  between  the  number  of  backorder-days  is  recorded;  call  the  result 

(say  the  number  of  backorder-days  with  the  extra  substitution  minus  the  back- 

order-days  without  it)  for  run  i . 

We  wish  to  determine  E(B^)  , since  if  E(B^)  > 0 we  should  not  perform 

the  additional  substitution  while,  if  E(B^)  < 0 we  should.  To  estimate 

n n 2 

E(B1 ) we  computed  Z B.  and  Z B.  ; we  could  then  estimate  E(B.  ) and 
A i=l  1 i=l  1 

V(B^)  and  construct  confidence  intervals  for  E(B^)  . Large  groups  of  runs 
were  made;  after  each  group  of  runs  a confidence  interval  was  computed  at  a 
selected  confidence  level.  If  this  confidence  interval  did  not  include  the 


origin  the  procedure  was  terminated  and  the  appropriate  strategy  was  selected 
as  optimal.  Moreover,  if  the  confidence  interval  did  include  zero  but  was 
shorter  than  some  pre-selected  tolerance  level,  the  procedure  was  terminated 
and  it  was  concluded  that  both  policies  gave  nearly  the  same  performance  level. 


(Actually  in  our  runs  this  outcome  did  not  occur. ) It  should  be  noted  that 
although  we  were  using  a sequential  procedure  we  used  analysis  appropriate 
for  a single  sample  and  that  this  is  not  precisely  correct.  However,  in 
each  case  the  results  obtained  by  simulation  were  the  same  as  those  obtained 
by  analytic  comparison.  Moreover,  the  simulation  was  computationally  superior 
in  two  respects:  the  amount  of  computer  time  did  not  grow  as  rapidly  with 
increased  system  size  as  it  did  for  the  analytic  method,  and,  perhaps  even 
more  relevant  in  terms  of  absolute  limitations,  the  simulations  did  not 
require  the  large  amounts  of  storage  necessitated  by  solving  such  large 
systems  of  equations.  Substitution  sets  (S)  generated  by  the  search  pro- 
cedure for  the  same  examples  presented  in  Section  III  are  found  in  Figures 
5,  6,  and  7. 

In  conclusion,  we  make  the  following  observations: 

1)  the  exact  solutions  obtained  through  linear  programming 
and  the  approximate  solutions  obtained  with  the  search 
support  the  assumption  that  the  form  of  the  optimal  policy 
satisfies  MR1)  - MR3); 

2)  comparison  of  exact  results  and  those  obtained  by  a search 
with  analytic  policy  evaluation  indicate  that  a search 

is  an  effective  method  policy  improvement; 

3)  the  correlation  between  results  of  search  with  analytic 
comparison  and  simulation  comparison  indicates  that  the 
simulation  method  is  valid  and  hence  gives  us  a method 
for  finding  an  approximation  to  an  optimal  policy  which 
can  be  applied  to  large  systems. 
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VI.  A SUMMARY  AMD  COMMENTS  CONCERNING  FUTURE  EFFORTS 

While  we  now  have  a method  for  policy  approximation  which  can  be  used 
on  large  systems,  there  are  several  possibilities  for  further  work  with  this 
model.  Continued  investigation  of  the  structure  of  an  optimal  policy  (i.e.. 
is  it  linear  in  any  of  its  variables?)  might  suggest  a reduction  in  the 
number  of  states  to  be  considered  for  inclusion  in  S during  a search  for 
an  approximation  to  the  optimal  policy.  A more  precise  comparison  of  the 
performance  of  the  exact  and  approximate  solutions  should  be  made  to  find 
a balance  between  reduced  backorders  and  computational  accessibility. 

In  addition,  we  plan  to  use  the  insights  obtained  through  the  study 
of  this  highly  simplified  inventory  model  as  a basis  for  our  future  efforts 
to  study  more  complex  situations. 

Unfortunately,  many  real-world  considerations  are  not  addressed  in  our 
simplified  model.  For  example,  we  have  ignored  the  fact  that  a)  sometimes 
a family  of  substitutable  items  may  consist  of  more  than  two  items,  b)  items 
are  normally  stocked  at  more  than  one  location,  c)  the  failure  and  repair 
distributions  may  not  be  stationary,  d)  the  planning  horizon  may  be  of  such 
a short  duration  that  infinite  horizon  models  may  be  inappropriate,  and 
e)  the  system  stock  level  for  each  item  may  not  remain  constant  for  an 
extended  period  of  time.  We  plan  to  address  many  of  these  issues  in  our 
future  work. 
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